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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above Is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days wilt be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 03 July 2003 , 
2a)M This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-5,8,10-16,20-22,24 and 28-36 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) [3 Claim(s) is/are allowed. 

6) 13 Claim(s) 1-5,8.10-16.20-22,24,26 and 28 is/are rejected. 
?)□ Claim{s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawjng(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )□ The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

1 3) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 -D Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) K Notice of References Cited (PTO-892) 4) □ Inten/lew Summary (PTO-41 3) Paper No(s). . 



2) □ Notice of Draflsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Infomial Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



The applicant's amendments to the specification and to the claims (specifically claim 15 with 
respect to the objection to the drawings) have overcome the objections to the specification and 
the drawings. Therefore, the objections are withdrawn. 



1 . Applicant's arguments filed 7/3/03 have been fully considered but they are not 
persuasive. 

With respect to the rejections over Narayanaswami et al in view of Epstein et al, the 
Applicant argues that the comparison, in the references, is applied to image data of the frames 
of data rather than to the video information. The applicant also argues that there is no 
motivation to compress the video information. The Examiner respectfully points out that the 
comparison being applied to the video information is a newly added limitation to the claims and 
will be addressed in the following office action. Furthermore, the claims do not recite the 
limitation of compressing the video information. The Examiner does not rely on the references 
to teach this compression, but rather watermarking image data. 

The Applicant further argues that Narayanaswami et al does not teach the video 
processing data comprises information for constructing three-dimensional models of objects in a 
scene of the video image data. The Examiner respectfully points out that this is a newly added 
limitation to the claims, and will be addressed in the following office action. 



2. Claim 1 1 is objected to because of the following informalities: it is unclear what the 
second "collected video information" is in the phrase "a comparator to compare a current state 
of the collected video information with collected video information" (lines 5-6 of the claim). 
Appropriate correction is required. 



Response to Arguments 



Claim Objections 
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Claim Rejections - 35 USC § 103 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-5, 8, 10-16, 20-22, 24, 28-31, and 35-36 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Narayanaswami et al (hereafter referred to as Narayanaswami) (US 
2003/0011684 Al). 

(Claims 1) Narayanaswami discloses a method of processing video data comprising the 
steps of accepting video frame data from a video source (page 2, section 0018; page 5, section 
0049, lines 14-17), accepting video information regarding the video frame data (page 5, section 
0049, lines 17-20), comparing a current state of the video information with gathered video 
information regarding the video frame data, and determining differential information based on 
the comparing (page 5, section 0051). Although Narayanaswami does not specifically disclose 
storing the differential information as annotations to the video frame data, it is disclosed that a 
parameter, such as location of the camera, is computed to determine its displacement, if any, 
from a previous position, so that the position of the camera may be accurately recorded (page 3, 
section 0035, lines 22-27). Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made that after comparing a current state of the video 



information (recorded parameters) with gathered video information, if differential information 
was determined to exist, such as if the location of the camera changed, then the differential 
information (e.g. new location of the camera) would be stored as annotations to the video frame 
data to ensure the correct parameter information (video information) would be accurately 
recorded. 
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(Claims 2-4) Narayanaswami further discloses the video information comprises camera 
geometry information, camera pose information, and source 

identification/description/illumination information (see the table on page 4, section 0043). 

(Claim 5) Narayanaswami further discloses the video frame data comprises images 
obtained from a camera (page 2, section 0018, section 0032, lines 6-8; page 5, section 0049, 
lines 14-17). 

(Claim 8) Narayanaswami discloses the parameters that are watermarked in the digital 
images captured by the camera include camera geometry information, camera pose 
information, and source identification/description/illumination information (see the table on page 
4, section 0043). Therefore, the step of comparing comprises comparing a current state with 
camera geometry information, camera pose information, and source 

identification/description/illumination information of the video frame data (page 5, section 0051). 

(Claims 10 and 30) See the rejection of claim 1 above. Furthermore, it would have 
been obvious that after the differential information has been stored as annotations to the video 
frame data the information would actually be appended to the video frame data so that the 
correct parameter information that was used when capturing the images would be accurately 
recorded on the frame data. 

(Claim 11) As best understood by the language of the claim, Narayanaswami discloses 
an apparatus comprising camera (100) that serves as a video source to generate video frame 
data, a collector configured to collect video information to be associated with the video frame 
data (page 3, section 0039; page 4, section 0044, lines 1-4), a comparator (218) to compare a 
current state of the collected video information with collected video information (see section 
0048), and a differential generator to determine differential information based on the comparison 
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(218, 220; page 5, sections 0048 and 0051). Regarding the annotator, see the rejections of 
claims 1 and 10 above. 

(Claim 12) See the rejection of claims 2-4 above. 

(Claims 13-14) Narayanaswami further discloses calibration software configured to 
generate the camera geometry information for the video frame data as the video frame data is 
being gathered by the video source and provide the generated camera geometry information to 
the collector, and pose estimation software configured to generate the camera pose information 
for the video frame data as the video frame data is being gathered by the video source and 
provide the generated camera pose information to the collector (page 3, sections 0034 and 
0035). 

(Claim 15) Narayanaswami further inherently discloses an encoder coupled to the 
differential generator configured to encode the differential information as an input to the 
annotator, the encoder being inherent in the translation of the differential information into the 
watermark signal. 

(Claim 16) Narayanaswami further discloses the encoder forwards a current state of the 
video information (e.g. new location of the camera) to a state storage device (in the watermark 
processor) coupled to the encoder. 

(Claims 20-22) See the rejection of claims 2-4 above and claim 29 below. 

(Claim 24) Narayanaswami further discloses the video source is a video capture device 
(section 0032, lines 1—8). 

(Claim 28) See the rejection of claim 10 above. 

(Claim 29) See the rejection of claim 1 above. Furthermore, Narayanaswami also 
discloses CPU (102) controls the operations of the camera via programs stored in a memory 
(108) and executed by the CPU (page 3, section 0033, lines 1-4). 
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(Claim 31) See the rejection of clainns 2-4 above. 

(Claim 35) Narayanaswami further discloses the video processing data comprises 
differential information indicating differences between video processing data for successive 
video image frames (sections 0048-0051). 

(Claim 36) Narayanaswami further discloses the video processing data is present only 
for video image frames for which differential information exists. The camera keeps track of 
which parameters should or should not be recorded at any given time, and if there is no change 
in the camera parameters between images, the parameters need not be present for those 
image frames, thus minimizing the processing load of the camera (section 0039). 

5. Claim 32 is rejected under 35 U.S.C. 103(a) as being unpatentable over Narayanaswami 
in view of Gloudemans et al (hereafter referred to as Gloudemans) (US 6597406 B2). 

Narayanaswami discloses an annotated video bit stream as discussed above, but does 
not specifically disclose a camera projection matrix defines the camera geometry information. 

Gloudemans discloses a system for enhancing a video presentation of a live event 
wherein a camera projection matrix defines the camera geometry information (col 24, line 65 - 
col 25, line 10). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to define the camera geometry information disclosed by Narayanaswami in 
the manner taught by Gloudemans, which is a common way to structure camera parameter 
information for use in data processing. 

6. Claims 33-34 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Narayanaswami in view of Matsumoto et al (hereafter referred to as Matsumoto) (US 
2002/0060686 Al). 
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Narayanaswami discloses an annotated video bit stream as discussed above, but does 
not specifically disclose the video processing data comprises information for constructing three- 
dimensional (3-D) models of objects in a scene of the video image data, and further wherein the 
video processing data comprises a 3-D scene model of objects in a scene of the video image 
data. 

Matsumoto discloses a 3-D model generating method and apparatus wherein the video 
processing data comprises information for constructing 3-D models of objects in a scene of the 
video image data, and further wherein the video processing data comprises a 3-D scene model 
of objects in a scene of the video image data (section 0007), 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to use the camera disclosed by Narayanaswami as a stereoscopic camera 
device for constructing 3-D models of objects in a scene of the video image data, and further 
wherein the video processing data would comprise a 3-D scene model of objects in a scene of 
the video image data, in the manner taught by Matsumoto, it being well known in the art to use a 
camera for constructing 3-D models of objects in a scene. 

Conclusion 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant 
is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
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will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Tia M Harris whose telephone number is 703-305-4807. The examiner 
can normally be reached on M-F 8:30 am - 6:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Christensen can be reached on 703-308-9644. The fax phone number for 
the organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-4700. 
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